Method and device for post-processing digital images

ABSTRACT

The invention relates to a method of post-processing digital images, the method comprising a step of frequency transformation TF ( 21 ), suitable for delivering a set of transformed coefficients (Y) from a set of pixels (y), extraction SEP ( 22 ) of original low-frequency coefficients (Ybfo) and original high-frequency coefficients (Yhfo) comprised in the set of transformed coefficients, correction COR ( 23 ), suitable for delivering a set of corrected transformed coefficients (Yc) from the original low-frequency coefficients, and combination ( 24 ), suitable for delivering a set of combined transformed coefficients (Yadd) from the original high-frequency coefficients and from the set of corrected transformed coefficients, and a step of inverse frequency transformation ITF ( 25 ) of the set of combined transformed coefficients, suitable for delivering a processed set of pixels (yout) that are ready for display on a screen.

FIELD OF THE INVENTION

[0001] The invention relates to a method and a device forpost-processing digital images comprising pixels.

[0002] The invention also relates to a computer program product forperforming such a post-processing method.

[0003] The invention notably finds its application in the correction ofdigital images that have previously been encoded and subsequentlydecoded in accordance with a block-based encoding technique, forexample, the MPEG standard, so as to attenuate visual artifacts causedby the block-based encoding technique.

BACKGROUND OF THE INVENTION

[0004] International patent application WO 97/29594 describes a methodand a device for post-processing decoded video data so as to minimizethe blocking artifacts in an image without affecting the contrast.

[0005] To this end, the method of post-processing data in accordancewith the prior art, described in FIG. 1, comprises the steps of:

[0006] LPF filtering (11) decoded video data (x), suitable fordelivering filtered data (xf),

[0007] DCT transform (12,13) of the filtered data and the decoded videodata, suitable for delivering transformed filtered data (Xf) andtransformed decoded data (X),

[0008] adjusting ADJ (14) the low-frequency coefficients comprised inthe transformed filtered data, suitable for delivering adjustedlow-frequency coefficients (Xbf),

[0009] combining (16) the high-frequency coefficients (Xhf) comprised inthe transformed decoded data and the adjusted low-frequencycoefficients, suitable for delivering combined transformed data (Xc),and

[0010] IDCT transform (17) of the combined transformed data, suitablefor delivering processed data (xc) that are ready for display on ascreen.

[0011] This method of post-processing decoded video data necessarilycomprises a low-pass filtering step for filtering the high-frequencycomponents. This method allows extraction, in the spatial domain, of thelow-frequency data on which a first DCT transform and an adjustment areapplied in order to deliver adjusted low-frequency coefficients. Thehigh-frequency coefficients are extracted (15) by way of a second DCTtransform of the decoded video data. Thus, one part of thedata-processing method is realized in the spatial domain, which rendersthe method both complex to be carried out and costly in terms ofcomputing.

[0012] Moreover, a low-frequency coefficient comprised in thetransformed filtered data is adjusted within the interval[Xq−q/2,Xq+q/2], where Xq is the value of said quantized low-frequencycoefficient and q is the value of the quantization step, so that such amethod necessitates access to the encoding parameters, which is notalways possible.

OBJECT AND SUMMARY OF THE INVENTION

[0013] It is an object of the present invention to provide a method ofpost-processing decoded video data, which can be performed in a simpleand economical manner.

[0014] To this end, the method of post-processing digital imagesaccording to the invention is characterized in that it comprises thesteps of:

[0015] frequency transformation, suitable for delivering a set oftransformed coefficients from a set of pixels,

[0016] extraction of original low-frequency coefficients and originalhigh-frequency coefficients comprised in the set of transformedcoefficients,

[0017] correction, suitable for delivering a set of correctedtransformed coefficients from the original low-frequency coefficients,and

[0018] combination, suitable for delivering a set of combinedtransformed coefficients from the original high-frequency coefficientsand from the set of corrected transformed coefficients.

[0019] In such a post-processing method, data are processed, andparticularly the original high and low-frequency coefficients areseparated, in the frequency domain. The post-processing method cantherefore be carried out in a simpler manner because the originallow-frequency coefficients are solely extracted and corrected via thetransformed coefficients. Consequently, neither the step of low-passfiltering in the spatial domain, nor a double frequency transformationof the filtered data and the decoded video data is necessary.

[0020] Said method is particularly adapted if the frequencytransformation step uses the same type of frequency transformation asthat used in the technique of block-encoding during a previous encodingof video data, for example, a DCT transform in the case of data thathave previously been encoded and subsequently decoded in accordance withthe MPEG standard. It also allows better control of the correction ofblocking artifacts.

[0021] In addition, the post-processing method is more effective,particularly when a first half of the set of transformed coefficientsconstitutes the original low-frequency coefficients and a second half ofsaid set constitutes the original high-frequency coefficients.

[0022] It is also a very flexible method as regards the restitution ofthe high frequencies in the image, wherein the combination step canlinearly combine corrected transformed high-frequency coefficients andoriginal high-frequency coefficients so as to deliver the set ofcombined transformed coefficients.

BRIEF DESCRIPTION OF THE DRAWINGS

[0023] These and other aspects of the invention are apparent from andwill be elucidated, by way of non-limitative example, with reference tothe embodiments described hereinafter.

[0024] In the drawings:

[0025]FIG. 1 illustrates the method of post-processing data inaccordance with the prior art,

[0026]FIG. 2 is a diagram showing the principal steps of the method ofpost-processing data according to the invention,

[0027]FIG. 3 illustrates a method of correcting blocking artifacts, and

[0028]FIG. 4 is a diagram showing an embodiment of the method ofpost-processing data according to the invention, comprising such a stepof correcting blocking artifacts.

DESCRIPTION OF PREFERRED EMBODIMENTS

[0029] The present invention relates to a method of post-processingdigital images that have previously been encoded and subsequentlydecoded in accordance with a block-based encoding technique andtherefore comprise blocking artifacts. As will be described hereinafter,the method of post-processing video data according to the invention isintended to determine:

[0030] the original high-frequency coefficients that must be preservedwithin a block of transformed coefficients so as to maintain the detailsof the image, such as contours, as well as

[0031] the original low-frequency coefficients that must be corrected soas to effectively suppress the blocking artifacts.

[0032]FIG. 2 shows diagrammatically the principal steps of the method ofpost-processing decoded video data according to the invention. Thismethod comprises the steps of:

[0033] frequency transformation TF (21), suitable for delivering a setof transformed coefficients (Y) from a set of pixels (y),

[0034] extraction SEP (22) of original low-frequency coefficients (Ybfo)and original high-frequency coefficients (Yhfo) comprised in the set oftransformed coefficients,

[0035] correction COR (23), suitable for delivering a set of correctedtransformed coefficients (Yc) from the original low-frequencycoefficients,

[0036] combination (24), suitable for delivering a set of combinedtransformed coefficients (Yadd) from the original high-frequencycoefficients and from the set of corrected transformed coefficients, and

[0037] inverse frequency transformation ITF (25) of the set of combinedtransformed coefficients, suitable for delivering a processed set ofpixels (yout) that are ready for display on a screen.

[0038] The set of pixels is preferably a segment of N pixels, with N=8in the case of the MPEG standard for which the encoding blocks generallycomprise 8 rows of 8 pixels. The set of pixels may be alternativelyconstituted by either an entire or a partial encoding block. Thefrequency transformation step preferably uses a transformation of theDCT type which is particularly adapted to the MPEG standard.

[0039] In the preferred embodiment, a segment of pixels is transformedin the transformation step into a segment of DCT coefficients.Subsequently, a first half of the segment of DCT coefficients, i.e. thefirst 4 DCT coefficients constituting the original low-frequencycoefficients (Ybfo), and a second half of the segment of DCTcoefficients, i.e. the last 4 DCT coefficients constituting the originalhigh-frequency coefficients (Yhfo), are extracted in the extractionstep. Such a separation into two parts of the segment of DCTcoefficients allows a better correction of blocking artifacts. It mayalso be easily adjusted optimally as compared with the prior-arttechnique, which necessitates the optimal adjustment of a low-passfilter from an infinite number of available filters.

[0040] The extraction step (22) also comprises a truncation sub-stepduring which the first 4 DCT coefficients are completed by 4 zerocoefficients so as to deliver a segment of truncated DCT coefficients.This truncation sub-step is in a way similar to a low-pass filteringoperation. In the correction step (23), the segments of truncatedcoefficients are then corrected so as to deliver a segment of correctedDCT coefficients comprising 4 corrected low-frequency DCT coefficients(Ybfc) and 4 corrected high-frequency DCT coefficients (Yhfc). In thecombination step (24), the 4 original high-frequency coefficients andthe segment of corrected DCT coefficients are combined so as to delivera segment of combined DCT coefficients (Yadd).

[0041] In a particularly advantageous embodiment, the segment ofcombined DCT coefficients corresponds to the concatenation of the 4corrected low-frequency DCT coefficients (Ybfc) and the 4 originalhigh-frequency coefficients (Yhfo). In the preferred embodiment, thesegment of combined DCT coefficients corresponds to the concatenation ofthe 4 corrected low-frequency DCT coefficients (Ybfc) and the 4 combinedhigh-frequency DCT coefficients, resulting in a linear combination ofthe 4 corrected high-frequency DCT coefficients (Yhfc) and the 4original high-frequency coefficients (Yhfo), namely:

Yhfadd=a.Yhfc+(1−b).Yhfo

[0042] where a and b are real values between 0 and 1, with, for example,a=½ and b=a/4 if Yhfc differs from 0 and b=½ if Yhfc is equal to 0. Sucha combination gives more weight to the original high-frequencycoefficients, if high frequencies appear in the corrected DCTcoefficients, and introduces an attenuation in the opposite case.

[0043] The segment of combined DCT coefficients is finally transformedin the spatial domain via an IDCT transform, which delivers a segment ofprocessed pixels (yout) for display on a screen.

[0044] The method of post-processing data also comprises at least ahorizontal processing of an image, associated with at least a verticalprocessing of said image. Indeed, the blocking artifacts may be presentat the borders of an encoding block, that is, in the four segmentsdelimiting the block vertically or horizontally. If the image isprocessed horizontally, vertical blocking artifacts will be corrected;conversely, horizontal blocking artifacts will be corrected if the imageis processed vertically. The method of post-processing data issuccessively applied to each of the two fields constituting a frame ifthe image is composed of two fields. It is preferably applied toluminance data in the digital image.

[0045] The correction step which is carried out is based on the numerousblocking artifact correction methods that are known to those skilled inthe art, and preferably those methods that do not use decodingparameters, which parameters are not always accessible.

[0046] In the preferred embodiment, the data correction method isreferred to as DFD method (DCT Frequency Deblocking). Such a datacorrection method comprises the following steps, shown in FIG. 3,namely:

[0047] a step of computing a first discrete cosine transform DCT1 (31)of a first segment (u) of N pixels, with N=8 in the example used,resulting in a first transformed segment U,

[0048] a step of computing a first discrete cosine transform DCT1 (32)of a second segment (v) of N pixels, which second segment is adjacent tothe first, resulting in a second transformed segment V,

[0049] a step of determining (33) a predicted maximum frequency (kpred)as a function of the maximum frequencies ku and kv of U and V, asfollows:

kpred=2.max(ku,kv)+2

[0050] with ku=max(kε{0, . . . ,N−1}/U(k)≠0),

[0051] kv=max(kε{0, . . . ,N−1}/V(k)≠0), and

[0052] max is the function giving the maximum of k among a set ofdetermined values,

[0053] a step of processing (35) a concatenated segment (w) comprising2N pixels, i.e. 16 pixels in our case, and corresponding to theconcatenation (34) of the first (u) and the second (v) segment, thisprocessing step comprising the following sub-steps of:

[0054] computing a second discrete cosine transform DCT2 (36) of theconcatenated segment (w), resulting in a transformed concatenatedsegment W,

[0055] correction (37) by setting those transformed data W(k) to zerothat have an odd frequency k which is higher than the predicted maximumfrequency (kpred), delivering a corrected transformed concatenatedsegment Wc,

[0056] computing an inverse discrete cosine transform IDCT2 (38) of thecorrected transformed segment Wc, delivering a corrected concatenatedsegment (cw).

[0057] In a preferred embodiment of the invention, filtering thresholdsare introduced in accordance with the following principle:

kumax=max(kε{0, . . . ,N−1}/abs(U(k))>T)

kvmax=max(kε{0, . . . ,N−1}/abs(V(k))>T)

[0058] where T is a threshold different from zero.

[0059] In the determination step (33), a more precise predicted maximumfrequency (kpred) is computed from the introduction of the threshold T,which allows a more effective correction of the blocking artifacts. Thevalue of the threshold T is a function of the size of the segments u andv. Indeed, a part solely consisting of pixels of the segments u and vmay be processed, for example, the pixels of either the even or the oddrows.

[0060] The correction step COR (37) preferably comprises a sub-step ofdetecting natural contours from the values of the pixels of the initialsegments u and v and the transformed segments U and V. This sub-stepallows distinction of the natural contours of the blocking artifacts. Anatural contour is detected if two conditions are met:

[0061] the average values of the pixels of the segments u and v on bothsides of a border between blocks differ by a considerable value which ishigher than a predetermined threshold M,

[0062] the segments u and v have a low spatial activity, which becomesmanifest by the fact that the values ku and kv are small and less than apredetermined value k0.

[0063]FIG. 4 is a diagram showing an embodiment of the method ofpost-processing data according to the invention, including the DFDmethod of correcting blocking artifacts as described with reference toFIG. 3. Such a method of post-processing video data comprises the stepsof:

[0064] DCT transform (41), suitable for delivering a first segment ofDCT coefficients (U) from a first set of pixels (u),

[0065] DCT transform (42), suitable for delivering a second segment ofDCT coefficients (V) from a second set of pixels (v) adjacent to thefirst segment of pixels,

[0066] extracting (43) first original low-frequency coefficients (Ubfo)and first original high-frequency coefficients (Uhfo) comprised in thefirst segment of DCT coefficients (U), suitable for delivering a firstsegment of truncated DCT coefficients (Ut) comprising the first originallow-frequency coefficients,

[0067] extracting (44) second original low-frequency coefficients (Vbfo)and second original high-frequency coefficients (Vhfo) comprised in thesecond segment of DCT coefficients (V), suitable for delivering a secondsegment of truncated DCT coefficients (Vt) comprising the secondoriginal low-frequency coefficients,

[0068] correction (23), suitable for delivering a first and a secondsegment of corrected DCT coefficients (Uc,Vc, respectively) from thefirst and second original low-frequency coefficients (Ubfo,Vbfo,respectively), comprising the sub-steps of:

[0069] IDCT transform (231) of the first segment of truncated DCTcoefficients (Ut), intended to deliver a first segment of a preprocessedpixel (ut),

[0070] IDCT transform (232) of the second segment of truncated DCTcoefficients (Vt), intended to deliver a second segment of apreprocessed pixel (vt),

[0071] DFD correction (230) of the segments of preprocessed pixels inaccordance with the principle described with reference to FIG. 3,suitable for delivering segments of corrected pixels (uc,vc),

[0072] DCT transform (233) of the first segment of corrected pixels(uc), suitable for delivering a first segment of corrected DCTcoefficients (Uc),

[0073] DCT transform (234) of the second segment of corrected pixels(vc), suitable for delivering a second segment of corrected DCTcoefficients (Vc),

[0074] combination (45), suitable for delivering a first segment ofcombined DCT coefficients (Uadd) from the first original high-frequencycoefficients (Uhfo) and the first segment of corrected DCT coefficients(Uc),

[0075] combination (46), suitable for delivering a second segment ofcombined DCT coefficients (Vadd) from the second original high-frequencycoefficients (Vhfo) and the second segment of corrected DCT coefficients(Vc),

[0076] IDCT transform (47) of the first segment of combined DCTcoefficients (Uadd), suitable for delivering a first segment ofprocessed pixels (uout) for display on a screen, and

[0077] IDCT transform (48) of the second segment of combined DCTcoefficients (Vadd), suitable for delivering a second segment ofprocessed pixels (vout), adjacent to the first segment of processedpixels (uout), for display on a screen.

[0078] The post-processing method described with reference to FIG. 4 hasthe advantage that it does not degrade the quality of the images that donot originally comprise any blocking artifacts. Indeed, in the presenceof such images, the original low-frequency coefficients are notsubjected to any modification because the DFD correction method used andthe combined high-frequency DCT coefficients remain identical to theoriginal high-frequency coefficients. In the prior-art technique, theuse of a low-pass filter in the spatial domain may, on the other hand,lead to a degradation of the quality of an image that does notoriginally comprise any blocking artifact.

[0079] The invention may be realized in the form of software embedded onone or several circuits performing the method of post-processing datadescribed hereinbefore, or in the form of items of hard ware. A devicefor post-processing digital images, corresponding to said method, isrepresented again by the functional blocks of FIG. 2. It comprises:

[0080] frequency transformation means TF (21), suitable for delivering aset of decoded transformed coefficients (Y) from a set of pixels (y),

[0081] means SEP (22) for extracting original low-frequency coefficients(Ybfo) and original high-frequency coefficients (Yhfo) comprised in theset of transformed coefficients,

[0082] correction means COR (23), suitable for delivering a set ofcorrected transformed coefficients (Yc) from the original low-frequencycoefficients, and

[0083] combination means (24), suitable for delivering a set of combinedtransformed coefficients (Yadd) from the original high-frequencycoefficients and from the set of corrected transformed coefficients, and

[0084] means ITF (25) for inverse frequency transformation of the set ofcombined transformed coefficients, suitable for delivering a set ofprocessed pixels (yout) that are ready for display on a screen.

[0085] Such a post-processing device can be inserted between a videodecoder and a television receiver in order to post-process decodeddigital images and to display the post-processed digital images on thetelevision receiver. Such a device can be built independently. It canalso be part of the video decoder or of the television receiver.

[0086] There are numerous ways of implementing the previously describedfunctions by means of software. In this respect, FIGS. 2 to 4 are verydiagrammatic. Therefore, although a Figure shows different functions inthe form of separate blocks, it does not exclude the fact that a singlepiece of software performs several functions. This neither excludes thefact that a function can be performed by a set of software. It ispossible to implement these functions by means of a suitably programmedvideo decoder circuit, a set top box, or a television receiver. A set ofinstructions comprised in a programming memory may cause the circuit toperform the different operations described hereinbefore with referenceto FIGS. 2 and 4. The set of instructions may also be loaded into theprogramming memory by reading a record carrier such as, for example, adisc comprising the set of instructions. Reading may alternatively beeffected through a communication network such as, for example, theInternet. In this case, a service provider will put the set ofinstructions at the disposal of those interested.

[0087] Any reference sign placed between parentheses in this text shallnot be construed as limiting the claim. Use of the verb “comprise” andits conjugations does not exclude the presence of elements or stepsother than those stated in the claims. Use of the article “a” or “an”preceding an element or a step does not exclude the presence of aplurality of such elements or steps.

1. A method of post-processing digital images comprising pixels, themethod comprising the steps of: frequency transformation (21), suitablefor delivering a set of transformed coefficients (Y) from a set ofpixels (y), extraction (22) of original low-frequency coefficients(Ybfo) and original high-frequency coefficients (Yhfo) comprised in theset of transformed coefficients, correction (23), suitable fordelivering a set of corrected transformed coefficients (Yc) from theoriginal low-frequency coefficients, and combination (24), suitable fordelivering a set of combined transformed coefficients (Yadd) from theoriginal high-frequency coefficients and from the set of correctedtransformed coefficients.
 2. A method of post-processing digital imagesas claimed in claim 1, wherein the extraction step (22) is suitable forextracting a first half of the set of transformed coefficients (Y) asbeing the original low-frequency coefficients (Ybfo) and a second halfof the set of transformed coefficients as being the originalhigh-frequency coefficients (Yhfo).
 3. A method of post-processingdigital images as claimed in claim 1, wherein the combination step (24)is suitable for linearly combining corrected transformed high-frequencycoefficients (Yhfc) and original high-frequency coefficients (Yhfo) soas to deliver the set of combined transformed coefficients (Yadd).
 4. Adevice for post-processing digital images comprising pixels, the devicecomprising: frequency transformation means (21), suitable for deliveringa set of decoded transformed coefficients (Y) from a set of pixels (y),means (22) for extracting original low-frequency coefficients (Ybfo) andoriginal high-frequency coefficients (Yhfo) comprised in the set oftransformed coefficients, correction means (23), suitable for deliveringa set of corrected transformed coefficients (Yc) from the originallow-frequency coefficients, and combination means (24), suitable fordelivering a set of combined transformed coefficients (Yadd) from theoriginal high-frequency coefficients and from the set of correctedtransformed coefficients.
 5. A video decoder adapted to deliver decodeddigital images and comprising a post-processing device as claimed as inclaim 4, suitable for post-processing the decoded digital images inorder to deliver post-processed digital pictures.
 6. A televisionreceiver adapted to receive digital images and comprising apost-processing device as claimed as in claim 4, suitable forpost-processing the digital picture in order to display post-processeddigital pictures on a screen of the television receiver.
 7. A computerprogram product for a video decoder, for example, comprising a set ofinstructions which, when loaded into the video decoder, cause this videodecoder to perform the method of post-processing digital images asclaimed in any one of claims 1 to 3.